Since the early 1990's, as the processing power of computers has increased, so has their ability to produce and present high-quality audio-visual content in the form of games, multimedia, animations, presentations and the like. With the Internet explosion of the late 90's, the ability of computers to share this content over a network and present it on websites has become increasingly important. Yet the tools for authoring, viewing, publishing and sharing of this content have evolved from disparate sources and specialized needs, and bring with them individual legacies and collective incompatibilities. Furthermore, the evolution of these tools has failed to keep pace with the growing mass market of computer users. In particular, many of the authoring tools were initially designed for professional use, and attempts to “dumb them down” for non-professional users have lead to mixed results.
In general, multimedia tools typically provide certain basic authoring and control features. Thes include means for importing and/or creating media objects (which can include images, sound, video, animated graphics, text and other data representative of audio-visual information), tools for editing the media objects, methods for enabling the author to specify the playback behavior of individual media objects within the overall presentation, and means for outputting finished multimedia content that can then be stored on Internet-enabled servers and linked together via the World Wide Web. All of these features vary from system to system, and the effective authoring and control of the applicable data invariably depends upon their successful implementation.
Regardless of the system involved, however, these features must be implemented according to the expected use of the system if they are to achieve the intended results. For example, the author may want to specify the circumstances under which a given media object starts and stops playing, its on-screen size and location, time-based or event-driven changes to its size and location, and various other playback behaviors appropriate to the object. The means by which this is done needs to be tailored to the expected skill level of the author. The features of an authoring system therefore vary greatly depending on how sophisticated the author is assumed to be.
In some systems, the author is assumed to be sophisticated. For example, if a professional web developer needs to create a cutting-edge, science-fiction-style user interface for a major entertainment website, extremely detailed control over the playback behavior of media objects is needed to provide the expected degree of activity and responsiveness. One method for achieving this level of detail involves the use of scripting, and is exemplified by Javascript used in conjunction with HTML in web applications.
Scripting methods operate at the most basic level by associating a list of instructions, called a script, with a triggering event. The possible triggering events for scripts vary, and include real-time input from the user interface, timers and clocks, trigger points within media object timelines, other scripts, etc. When the triggering event for a script occurs, a script playback engine, called an interpreter, reads the instructions in the script and performs the actions and procedures specified therein. In a multimedia system, these actions and procedures not only provide the features customarily found in any programming environment, such as sub-routines, conditional branches, general purpose variables, math operations, etc., but they also include facilities for manipulating and controlling the playback of media objects.
Regardless of the specific instructions supported by a particular scripting method, the results obtained from running the script must reflect the author's intentions. However, current methods of scripting are often unsatisfactory due to the procedural nature of scripting. Especially when dealing with highly interactive multimedia presentations with numerous possible states and paths, the step-by-step nature of the script instructions tends to obscure the true structure of the content, and requires substantial mental visualization, i.e. sophistication on the part of the author, to be used effectively. As a method for controlling the behavior of media objects in a multimedia system where ease of use for unsophisticated users is needed, current methods of procedural scripting have significant drawbacks.
One drawback is that the procedures used to create the scripts are abstract and fragmented, and do not correspond either directly or intuitively to the shape and structure of the resulting multimedia presentation. This problem becomes worse as the size and complexity of the multimedia presentation increases. To create a set of scripts that represents a highly interactive or expansive presentation, a large number of non-intuitive, interrelated procedures must be performed, often presenting a complex or frustrating burden to authors, particularly those who are inexperienced.
Another drawback is that current methods of scripting frequently result in rigid or too simplistic algorithms for determining the activity in a presentation. This tendency often leads to mechanical-looking results and misses the objective of creating appealing and entertaining multimedia content. For systems that otherwise derive advantage from creating multimedia content that is attractive or interesting to a viewing audience, this presents a burden in delivering appropriate results.
Therefore, current systems and methods which use scripting to enable the author to specify the playback behavior of media objects are generally unable to provide the clarity, intuitiveness, and ease of use required to efficiently and effectively produce multimedia, websites, and other audio-visual content.
Some authoring systems are designed based on the assumption that the author is unsophisticated. For example, when a consumer wants to present a slide-show of family photographs on a personal website, only minimal control over the playback behavior of media objects is needed (i.e., selection of the order of the photographs), but the process of specifying that control must be simple enough to be easily understood by the average consumer. One method for enabling the author to specify the playback behavior of media objects where the manner of specifying is simple and easy to understand involves the use of templates.
As a way of enabling the author to specify the playback behavior of media objects where the manner of specifying is simple and easy to understand, template methods in this field operate at the most basic level by providing means for the author to associate individual media objects or sets of media objects with specified roles or events within a template, or piece of multimedia content in which the playback behavior of the media objects has been programmed in advance but in which some of the media objects have been left unspecified. Once the author has assigned one or more media objects to their desired roles or events within the template, the system incorporates the specified media objects into the template and outputs the combination as a finished piece of multimedia content.
Regardless of the specific means provided for associating media objects with roles or events within a template, the combination of author-selected media objects and pre-programmed template must produce the desired results by displaying multimedia content that reflects the creative intentions of the author. However, current methods that use templates are frequently unsatisfactory due to the pre-programmed nature of the template, and because the author generally has no input into the form of the template or the functionality it provides. Particularly with regard to multimedia presentations where individuality or uniqueness is desirable, an author's creative intentions are frequently unable to be realized using templates. As a method for controlling the behavior of media objects in a multimedia system where detailed or flexible control is needed, current methods using templates have significant drawbacks.
One drawback with current methods using templates is that the procedures undertaken to develop templates cannot take into account each individual author's profile, likes and/or dislikes, but must rather make broad assumptions about authors in general. For templates to be effective in assisting authors to realize their creative intentions, this presents a burden in the provision of effective assistance.
Another drawback is that current methods using templates typically result in media object behavior that is generic, inflexible or overly simplistic. This tendency often leads to multimedia content which is “canned” looking or otherwise uninteresting. For systems that derive advantage from creating multimedia content that has such characteristics as uniqueness or interest for the viewing audience, this presents a burden in delivering appropriate results.
Therefore, current systems and methods using templates to enable the author to specify the playback behavior of media objects are generally unable to provide the usefulness, flexibility, and author-specific objectives required to efficiently and effectively enable satisfactory authoring of multimedia, websites, and other audio-visual content.
Some systems focus on the passage of time in how they organize the playback behavior of media objects. For example, when a consumer wants to shoot several minutes of home video and use it to create a one minute ‘movie,’ the ability to take a number of separate media elements, such as video segments, sound effects, musical clips, text items, etc., and organize them in time is required. One way to provide this ability this is through the use of timelines.
Timeline methods in this field operate at the most basic level by providing means for the author to place representations of media objects on a line which is divided into frames, each frame representing a discrete, constant unit of time passing. The “frame rate” of a timeline specifies the length of this constant unit; for example, a frame rate of ten frames per second means that each unit is a tenth of a second long.
In professional timeline authoring tools such as Macromedia's Flash 5, the timeline may further be divided into independent data paths or “layers” which allow the author to develop the behavior of several media objects in parallel. For example, one layer could be used for behavior changes to a background object, and speparate layers might be used for each of several foreground objects. Additional means may be provided for organizing timelines into interconnected “scenes”, looping timelines, and nesting timelines within timelines, and exporting the results as completed “movies.” In professional authoring tools such as Flash 5, scripting capabilities may be provided so that the author can program timelines and their contents to behave interactively according to end-user input.
Typically, finished movies are played back using a special program, or in the case of web playback of Flash 5 files, using a web browser plug-in. The plug-in or program interprets the movie data on a frame-by-frame basis and displays it to the end user in a playback display.
Regardless of the specific means for defining playback behavior in time, the timeline approach must produce the desired results in a variety of situations. For example, the author may be unsophisticated, or need to produce content very quickly, or the content being produced may need to be highly interactive for the end user. In these cases, the timeline approach has significant drawbacks.
If the user is unsophisticated, or the content must be produced very quickly, the timeline approach represented by tools such as Flash 5 is daunting. Defining behavior on a frame-by-frame basis is an inherently painstaking process, not conducive to quick gestures or spur-of-the-moment ideas. For example, a simple idea like “fade this picture out for three seconds” becomes “place a picture at this frame, now place it 36 frames later, now set its properties at the first frame, now set them at the later frame, now create a motion ‘tween’ (transition object) between them.”
When interactivity is required the situation gets even more complex. Timelines are inherently linear in that they represent pre-defined behavior over time. But interactive presentations are inherently non-linear, since they must account for circumstances and user choices that will happen at unpredictable times. Therefore, in order to program even the most basic levels of interactivity, professional tools like Flash 5 must employ scripting, which has drawbacks that have already been discussed.
Some tools providers have addressed the problems of complexity and interactivity through the use of timeline templates. These templates are typically canned sequences produced by professionals that can be laid out, interconnected and adjusted somewhat by the author; for example, the author may be able to specify some color schemes, fonts and custom text that appears in the animations. Examples of this approach are found in the sites Moonfruit.com and Electrifier.com. But timeline templates present similar problems to the templates already discussed. They are inflexible, limit the creative input of the user and often result in a canned look.
Regardless of the method used to enable the author to specify the behavior of media objects, outputting a finished piece of multimedia content frequently involves exporting it to a computer file which can then be stored on an Internet-enabled server and linked with other files via the World Wide Web. For example, someone using a web authoring tool to create a personal home page will typically output the page as a browser-readable HTML file and store it on a Web server, at which point the author's friends and family can view the page in their browsers and can put links to it in their own home pages. The linking method provided by the World Wide Web allows separate pieces of multimedia content located on different computers and authored by different people to be connected together into a web by including within one piece of multimedia content a pointer to the Web address of another.
This linking method provided by the World Wide Web operates at the most basic level by defining a browser-readable format for Web content called HTML (which may be extended by Javascript and downloadable browser “plug-ins” that can read and play custom formats), and by allowing authors to associate a triggering event in one piece of Web content (such as the user clicking on a particular line of text displayed in his or her browser) with the Web address of a second piece of Web content. When the triggering event occurs, the browser discards the current piece of Web content and loads and displays the second piece of Web content, effecting a page change. The address specified in the link identifies the desired Web content in terms of the Web server on which it is stored, using either the Internet Protocol (IP) address of the server or a registered domain name (such as www.yahoo.com) that is converted into the IP address by a Domain Name Server (DNS). Regardless of the type of authoring system used to create multimedia content for the World Wide Web, the system typically depends on this linking method to interconnect pieces of multimedia content stored on different servers.
The linked pages thus created and posted to Web servers are then viewed in a browser, which is a playback program that allows people to traverse, or navigate, through the distributed content page by page. This navigation process is distinguished in this document from “exploration” of files on the user's hard drive in that exploration involves traversing through a hierarchy of files, usually presented in a GUI using a data display with a desktop metaphor, as is done in Microsoft ® Windows ®. This data display represents to the user the current location in the hierarchy being examined and provides means to further explore up or down levels in the hierarchy, by opening nested containers called directories or folders and presenting their contents. It is relatively easy for a user to understand at any time where a particular folder or file is in the hierarchy.
Navigation, on the other hand, does not require a hierarchy. Navigation displays directly the content of a file or object being examined and maintains information about the file or object that was previously examined, providing a “back” function to return to the contents of the previous file or object. A “forward” function may further be provided to return to the most recent content that a “back” function has been executed from. In other words, when navigating the web, there is no absolute sense of where one is, only the most recent snapshot of where one has been.
Consequently, the method used by Web browsers to effect page changes is conceptually simplistic, involving first discarding the current page and then loading and displaying the next, and this can be inefficient or otherwise problematic in terms of the “Web surfing” experience of users. Particularly when it comes to effectively presenting link destinations, the smoothness of transitions between Web pages, and the robustness of links when Web content is moved, the linking method of the World Wide Web and its implementation by Web browsers have significant drawbacks.
One drawback of the linking method of the World Wide Web is that the procedures a Web browser uses to access data for a new page do not begin until the triggering event for the link has occurred. This means that after clicking on a link, the user must wait for the data of the new page to load before seeing the new page, which often takes a noticeable amount of time, Furthermore, it means that the user cannot preview the actual destination of the link prior to committing to the page transition, but can only evaluate the destination based on information provided in the current page, which is frequently inadequate. As a result, users frequently become frustrated after triggering links which are slow to load, irrelevant to the purposes at hand, or both. This presents a substantial burden in providing an efficient and pleasing Web navigation or “surfing” experience for the user.
Another problem with the linking method of the World Wide Web is that the Web addresses used in links identify pieces of Web content in terms of the server hosting the data rather than the data itself. This means that if a piece of Web content is moved to a different server, any existing links to that Web content will become invalid, resulting in “broken links.” This approach misses the objective of providing Web links that are as robust and flexible as possible.
Therefore, current authoring and playback systems which depend on the linking method of the World Wide Web to provide linking of content across different servers inherit the drawbacks of that method, which include the inability to effectively represent link destinations to users, the inability to implement page changes with the smoothness, efficiency and flexibility required to provide a fully satisfactory Web surfing experience for users, and the inability to reference Web content when that content is moved.
Considered separately, current systems for authoring, publishing, sharing and viewing of multimedia content present drawbacks to the user. What is needed in authoring systems is a level of control over the content provided by scripting, without the complexity, loss of perspective and mechanical results inherent in scripting methods. What is further needed in authoring systems is a simplicity and ease of use found in templates, without the restrictions imposed by template methods. What is also needed is the ability to control time-based and interactive processes without the inflexibility imposed by timelines. What is needed in publishing, sharing and viewing systems is a manner of interconnecting content distributed on the Web or other network that allows for a smooth and pleasing navigation experience for the user and is robust when portions of the content are moved from one location to another.
Finally, when taken together, current systems for authoring publishing and sharing present a collective burden in that: 1) the number and quality of tools varies greatly; 2) each only addresses a specific part of a much larger picture; and 3) incompatibilities and inefficiencies arise as content is passed from one tool to the next along the production, publishing and viewing chain. Therefore what is further needed is a unified approach to authoring, publishing, sharing and viewing that eliminates these inefficiencies and incompatibilities.